Papers with syntactic metrics
An Error Analysis Framework for Shallow Surface Realization (2021.tacl-1)
Copied to clipboard
| Challenge: | BLEU and METEOR metrics fail to provide information on which linguistic factors impact performance of natural language generation models. |
| Approach: | They propose a framework for error analysis which permits identifying which features of the input affect the models’ results. |
| Outcome: | The proposed framework improves the performance of 174 system runs submitted to the Multilingual SR shared tasks. |
Evaluating Readability Metrics for German Medical Text Simplification (2025.coling-main)
Copied to clipboard
| Challenge: | Clinical reports and scientific information sources are written for medical experts, preventing patients from understanding the main messages of these texts. |
| Approach: | They evaluated the suitability of 18 statistical, part-of-speech-based, syntactic, semantic and fluency metrics to measure readability of German medical texts. |
| Outcome: | The proposed measures are compared with standard methods on English medical texts and simplified summaries. |